Correct for Whom? Subjectivity and the Evaluation of Personalized Image Aesthetics Assessment Models

The problem of image aesthetic quality assessment is surprisingly difficult to define precisely. Most early work attempted estimate the average rating a group observers, while some recent has shifted an approach based on few-shot personalization. In this paper, we connect personalization, via Immanuel Kant's concept disinterested judgment, argument from feminist aesthetics about biased tendencies objective standards for subjective pleasures. To empirically investigate philosophical debate, introduce PR-AADB, relabeling existing AADB dataset with labels pairs images, and measure how well groundtruth predicts our new pairwise labels. We find, consistent critique, that both personalized predictions represent users' preferences significantly better than others, but it predict when whom will be correct. thus advise against using benchmark datasets evaluate models IAQA, recommend caution attempting account difference machine learning more generally.